Taylor Swift Lyrics Analysis: Is She Way Too Lovery-dovery in Her Songs?

1. Gather Data

For data collection from Genius, we use the Cran package “geniusr,” which provides loads of functions to get lyrics and other music info. We first register a genius token for permission, get all Taylor Swift songs, merge the table with album names, and use both song id and the URL of lyrics to retrieve the lyrics. There is an issue when we get lyrics: two lyrics data frames generated from song-id and lyrics URL have overlapping rows but are not precisely the same; so we take song_id as the primary key, take the rows of difference as a subset from one data frame, and concatenate the subset to the second data frame. In this way, we obtain a complete data set, with Taylor’s all songs, album names, and lyrics in lines.

data retrive and merge

Next, we clean the raw data and get it ready for analysis. The data table contains unnecessary rows such as Taylor’s speeches, interviews, voice memos, and liner notes. We find these patterns in song names and remove rows. We count the number of words in each line and calculate song length by grouping song id. Also, we remove songs with a size less than 60 characters (mostly youtube advertisements). We create a new column called collaborator by extracting section artist names from another column, using regex. Ultimately, we replace all empty cells with N/As. In this clean data table, there are 15426 observations and 11 variables. We will first focus on exploratory data analysis and then use the lines of the lyrics for text analysis. What worth paying attention to is, each row means one line of song lyrics; in this way, the song id is no longer a primary key to distinguish each row. We will use a lot of the “unique()” function to remove duplicates in rows for this issue.